Different aspects of expert pronunciation quality ratings and their relation to scores produced by speech recognition algorithms

The ultimate aim of the research reported on here is to develop an automatic testing system for Dutch pronunciation. In the experiment described in this paper automatic scores of telephone speech produced by native and non-native speakers of Dutch are compared with speci®c, i.e., temporal and segmental, and global pronunciation ratings assigned by three groups of experts: three phoneticians and two groups of three speech therapists. The goals of this experiment are to determinutee (1) whether speci®c expert ratings of pronunciation quality contribute to our understanding of the relation between human pronunciation scores and machine scores of speech quality, (2) whether di€erent expert groups assign essentially di€erent ratings, and (3) to what extent rater pronunciation scores can be predicted on the basis of automatic scores. The results show that collecting speci®c ratings along with overall ones leads to a better understanding of the relation between human and automatic pronunciation assessment. Furthermore, after normalization no considerable di€erences are observed between the ratings by the three expert groups. Finally, it appears that the speech quality scores produced by our speech recognizer can predict expert pronunciation ratings with a high degree of accuracy. Ó 2000 Published by Elsevier Science B.V. All rights reserved.